When a subject is photographed by a video camera under an illumination of a fluorescent lamp directly operated from commercial AC power, chronological brightness level variations in a video signal as an image output, i.e., a fluorescent light flicker occurs due to a difference between the frequency (twice the frequency of the commercial AC power) of a luminance change (light intensity change) of the fluorescent lamp and a vertical synchronization frequency of the camera.
For example, in a zone where the frequency of the commercial AC power is 50 Hz, a subject may be photographed by a CCD camera of an NTSC method (at a 60 Hz vertical synchronization frequency) under the illumination of the non-inverter fluorescent lamp. In such a case as shown in FIG. 28, one field frequency is 1/60 second while the period of the luminance change of the fluorescent lamp is 1/100 second. The exposure timing at each field drifts with respect to variations in the luminance of the fluorescent lamp, and an amount of exposure light at each pixel changes from field to field.
If the exposure time is 1/60 second, the amount of exposure light is different within the same exposure time from duration a1 to duration a2 to duration a3, and when the exposure time is shorter than 1/60 second (but not 1/100 second), the amount of exposure light time is different within the same exposure from duration b1 to duration b2 to duration b3.
Since the exposure timing responsive to the luminance change of the fluorescent lamp reverts back to the original timing every three fields, the brightness level variation due to flickering is repeated every three fields. More specifically, the luminance ratio of each field (the appearance of a flicker) changes within an exposure period, but the period of flickering remains unchanged.
In a progressive camera such as a digital camera, the brightness level variation is repeated every three frames if the vertical synchronization frequency is 30 Hz.
To emit white light, a plurality of fluorescent lamps, for example, a red fluorescent lamp, a green fluorescent lamp, and a blue fluorescent lamp are typically used. Fluorescent materials for these lamps have unique persistence characteristics thereof, and for a duration of time from a stop of discharging to a subsequent start of discharging, light emission decays in accordance with the persistence characteristics. During this duration of time, light appearing white at first decays while the hue thereof changes at the same time. If the exposure timing drifts, not only the brightness level variations but also the hue change occurs. Since the fluorescent lamp has unique spectral characteristics that a strong peak is present in a particular wavelength, a variable component of a signal becomes different from color to color.
The color hue change and the difference in the variable component from color to color lead to a so-called color flicker.
As shown in the bottom portion of FIG. 28, the amount of exposure light remains constant regardless of the exposure timing if the exposure time is set to be an integer multiple of periods ( 1/100 second) of the luminance variation of the fluorescent lamp. No flicker then occurs.
It is contemplated that illumination of the fluorescent lamp is detected through signal processing of a camera in response to an operation of a user, and that the exposure time is set to be an integer multiple of 1/100 second under the illumination of the fluorescent lamp. In this arrangement, a simple method can fully control the generation of the flickering.
However, since this method does not allow the exposure time to be set to any value, the freedom of the exposure amount adjustment means for obtaining an appropriate amount of exposure is reduced.
A method for reducing the fluorescent light flicker under any shutter speed (exposure time) is thus required.
An image pickup device having all pixels in one frame exposed at the same exposure timing, such as a CCD image pickup device, offers relatively easily such a method because the brightness level variations and color variations due to the flickering appears only between fields.
If the exposure time is not 1/100 second, the flickering has a repetition frequency of three fields as shown in FIG. 28. To achieve a constant average value of the video signal of each field, current luminance variations and current color variations are predicted from the video signal three fields before, and the gain of the video signal of each field is adjusted based on the prediction results so that the flicker is reduced to a level that presents no problem in practice.
In the XY addressing type scanning image pickup device, such as a CMOS image pickup device, however, the exposure timing drifts successively from pixel to pixel by one horizontal period of the reading clock (pixel clock) in a screen horizontal direction. Since all pixels are different in the exposure timing, the above-referenced method cannot suppress the flickering.
FIG. 29 illustrates such flickering. The pixels successively drift in the exposure timing in the screen horizontal direction as discussed above. One horizontal period is sufficiently shorter than the period of the variation of the fluorescent light. Based on the assumption that the pixels on the same line have the same timing, the exposure timing of each line in a screen vertical direction becomes something like the one shown in FIG. 29. This assumption presents no practical problem.
The exposure timing is different from line to line in the XY addressing type scanning image pickup device, such as a CMOS image pickup device as shown in FIG. 29 (F1 represents such a drift in the exposure timing). Since the lines suffer from a difference in the amount of exposure light, the brightness level variation and the color variation take place due to the flickering not only between fields but also within each field. A strip pattern appears on a screen (with strips thereof aligned in the horizontal direction and the density of the stripes changing in the vertical direction).
FIG. 30 illustrates an on-screen flicker if a subject is a uniform pattern. Since one horizontal period (one wavelength) of the subject is 1/100 second, a stripe pattern of 1.666 periods appears in one frame. Let M represent the number of read lines per field, and one horizontal period of the stripe pattern corresponds to the number of read lines L=M*60/100. In this description and the drawing, an asterisk (*) represents a multiplication operation.
As shown in FIG. 31, three fields (three frames) correspond to five periods (five wavelengths) of the stripe pattern, and if viewed continuously, the stripe pattern appears to drift in a vertical direction.
FIG. 30 and FIG. 31 show only the brightness level variation due to the flicker. In practice, however, the above-described color variation also additionally appears, thereby substantially degrading image quality. The color flicker, in particular, becomes pronounced as the shutter speed becomes fast. The XY addressing type scanning image pickup device suffers more from image quality degradation because the effect of the color flicker also appears on the screen.
If the exposure timing is set to be an integer multiple of periods ( 1/100 second) of the luminance variation of the fluorescent light, the amount of exposure becomes constant regardless of the exposure timing, and a fluorescent light flicker containing an on-screen flicker does not occur.
With a variable electronic shutter speed feature incorporated, a CMOS image pickup device becomes complex in structure. Even in an image pickup device having the electronic shutter, the flexibility of the exposure amount adjusting means for achieving an appropriate exposure is reduced if only an integer multiple of 1/100 second is set as the exposure time to prevent flickering.
Methods for reducing the fluorescent light flickering for use in the XY addressing type scanning image pickup device, such as the CMOS image pickup device, have been proposed.
Patent document 1 (Japanese Unexamined Patent Application Publication No. 2000-350102) and patent document 2 (Japanese Unexamined Patent Application Publication No. 2000-23040) discloses methods of estimating a flicker component by measuring an amount of light of a fluorescent lamp with a photosensitive element or a measuring element and controlling a gain of a video signal from an image pickup element in response to the estimation result.
Patent document 3 (Japanese Unexamined Patent Application Publication No. 2001-16508) discloses another technique. In accordance with the disclosed technique, two types of images are taken in two conditions, namely, a first electronic shutter value appropriate for a current ambient illumination condition and in a second electronic shutter value having a predetermined relationship to a light and dark cycle of a fluorescent lamp, a flicker component is estimated by comparing the two signals, and a gain of a video signal from an image pickup device is controlled in response to the estimation results.
Patent document 4 (Japanese Unexamined Patent Application Publication No. 11-164192) discloses another technique. In accordance with the disclosed technique, a brightness variation under an illumination of a fluorescent lamp is recorded beforehand as a correction factor in a memory, the phase of a flicker component is detected from a video signal from an image pickup device taking advantage of a difference between the frequency of a video signal component and the frequency of the flicker component, and the video signal is thus corrected in accordance with the correction factor in the memory in response to the detection results.
Patent document 5 (Japanese Unexamined Patent Application Publication No. 2000-165752) discloses another technique. In the disclosed technique, a correction coefficient is calculated from two video signals that are obtained as a result of exposures performed with a time difference, the time difference causing the phase of flicker to be inverted by 180 degrees.
As disclosed in patent documents 1 and 2, the technique of estimating of the flicker component by measuring the amount of light of the fluorescent lamp with the photosensitive element or the measuring element increases the size and the cost of the image pickup system because the photosensitive element or the measuring element is attached to the image pickup device.
As disclosed in patent document 3, the technique of estimating the flicker component by photographing the two types of images in the different shutter conditions (exposure conditions) requires a complex system in the image pickup device, and further this technique is not appropriate for taking a moving image.
The technique disclosed in patent document 4 uses the coefficient prepared beforehand in the memory as a correction signal. It is practically impossible to prepare the correction coefficients for all types of fluorescent lamps. Depending on the type of the fluorescent lamp, detecting accurately the flicker component and reducing reliably the flicker component are difficult. As disclosed in patent document 4, the technique of extracting the flicker component from the video signal taking advantage of the difference between the frequencies of the video signal component and the flicker component has difficulty in detecting the flicker component distinctly from the video signal component in a black background portion and a low-illuminance portion, each portion having a small amount of flicker component. If a moving image is present in a screen, performance for detecting the flicker component is substantially lowered.
As the technique disclosed in patent document 3, the technique disclosed in patent document 5 for estimating the flicker component by photographing the two types of images at the different timings requires a complex system in the image pickup device and is not appropriate for taking a moving image.
In accordance with the present invention, a fluorescent light flicker characteristic of an XY addressing type scanning image pickup device such as a CMOS image pickup device is accurately detected and reliably and sufficiently reduced through simple signal processing without using an photosensitive element regardless of the level of a video signal of a subject and the type of a fluorescent lamp.